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ABSTRACT 


The purpose of this study was to test the effectiveness of 
five different methods of teaching paired-associates on the IBM 1500. 
Effectiveness was measured by two different criteria tests - a 
recall test and a recognition test. Each of these criterion tests 
was used to measure the effectiveness of the methods over two dif- 
ferent time periods - immediately after the teaching sessions and 
one week after the teaching session. The criterion used was amount 
learned per unit time. Sixty-eight, twelve to fourteen year old 
subjects were randomly assigned to the five teaching methods and two 
criterion test types. 

The results of this experiment, though not significant, 
would tend to indicate that the Prompt methods of teaching paired~- 
associates was the most effective of those tried if one considers 
short term effects. If long term effects or retention are consi- 
dered in relation to recall tasks, there appears to be little dif- 
ferential effect among the five methods tested. Recognition tasks 
on the other hand tend to remain more consistent over time and thus 
the Prompt method would be considered superior for this type of task 
(at least for short term retention tasks - i.e. one or two weeks). 

No interaction between the constructed response and recog- 
nition criterion tests was found, indicating that the teaching methods 
did not appear to differentially affect the subjects’ achievement 
as measured by two different procedures. 

The S-R method of teaching, though resulting in the second 


highest mean criterion score, had the lowest return rate for the 
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second session, indicating that the motivation factor cannot be ig- 


nored in any teaching method chosen in C.A.I. 
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CHAPTER I 
THE PROBLEM, ITS NATURE AND SIGNIFICANCE 


I. INTRODUCTION 


When the computer was first combined with the typewriter, 
cathode ray tube and other sensing devices it marked the beginning 
of a new era in the development of education. This new era may be as 
far reaching and drastic a change as to rival any of the great social 
revolutions of the past. The application of computers to instruction 
is growing so fast it makes papers written as recently as six years 
ago obsolete. The application which is the focus of the present 
study has popularly been called "Computer Assisted Instruction" 
(C.A.I.) and is not to be confused with programmed learning which 
generally refers to programmed text books or mechanical teaching 
machines of the Skinner type (Skinner 1958). 

The faith and hope for the future where learning will be in- 
teresting for everyone, where failure will vanish and where all students, 
rich or poor, will have equal opportunities to develop their potential, 
lies in the hands of the educators who are willing and able to ad- 
just the educational system on the basis of empirical evidence. 

They must design the system in such a way that empirical feedback is 
always available for decision making. One of the ways this seemingly 
impossible task may be approximated is through the intelligent imple- 
mentation and use of the more sophisticated computers which are now 
available and will be developed in the future. This project is one of 


the many which will be needed if C.A.I. is to advance and indeed if 
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education is to advance. What makes research of this nature mandatory 
in the C.A.I. field is the need to optimize the efficiency of any such 
teaching system so that the cost per student taught will become much 
lower and eventually competitive with conventional systems. The 

world economic and ideological race as well as the information explosion 
is further incentive to increase our schools' efficiency. In this 
author's opinion the computer has a very definite major role to play 


in this respect. 
II. THE PROBLEM AND ITS IMPORTANCE 


The main purpose of this study was to investigate ways of 
improving the efficiency of automated teaching. More specifically, 
an attempt was made to discover ways of increasing the efficiency of 
teaching on the IBM 1500 computerized teaching machine through a study 
of five strategies of teaching paired associates. 

The methods of teaching the paired associates were derived 
partly from the programmed learning field and partly from the writer's 
and his associates' experience in the C.A.I. area. Four of the five 
methods parallelled as closely as possible the methods described in 
the literature on programmed learning with the exception that they 
were adapted for presentation on the computer (the ultimate objective 
being that no human would have to intervene during the entire teaching 
and testing process). The remaining method was one which is uniquely 
a C.A.I. method in that no mechanical teaching machine or programmed 
text book could teach in a similar way. 

There are three main sections to this study: 


1. An investigation into the differential effect of the 
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teaching methods as studied over different criterion 
measures. 

2. A study of the differential effect of the teaching methods 
as affected by time and as measured by different criterion 
tests. 

3. An attempt to relate the procedure and results to current 
learning theory. 

A rationale for choosing the investigation of the teaching 

of paired-associates as an important indication of a general teaching 
situation may be presented as follows. It may be possible to generalize 
from the teaching of paired-associates because some of what is now 
taught in school is of the paired-associate type or may be broken 

down into a variation of paired-associate learning. For example a 
concept such as the relative mass of objects composed of different 
materials is associated with the word specific gravity. Kinematics 

is associated with a particular branch of physics and elicits a dif- 
ferent response than does the stimulus light. When one wants to learn 
a new word he looks in the dictionary and is given a list of other 
words with which he is to associate the word in question and with 

which, hopefully, new meaning will be associated. One "knows a subject" 
or "has learned a subject" if he can use the shorthand, the code or 

the vocabulary of that subject. The need for such a code or abbreviated 
way of saying things arises out of the need to communicate vast numbers 
of paired-associates in an instant. For example a race driver, who 
when trying to communicate with his mechanic, would say "the rate of 


change of position of my car with respect to time squared is too small," 
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(instead of "not enough acceleration") would still be in the starting 
position when the winner crossed the finish line. The assumption 
will thus be made that an important facet of learning is the pairing 
of sounds or labels with objects and with constructs. The sound 
"kar" is associated with a four wheeled metallic vehicle which ingests 
people, moves off, then spits them out. Or it is associated with 
marks on a piece of paper which look like "car". This same sound 
may eventually become associated with girls or the hypothetical con- 
Struct. tan: 

Skinner's learning theory and his discussion of the utility 
of teaching machines is based on the assumption that everything can 
be broken down into simple enough terms that virtually all of the 
people trying a particular frame (or unit of learning) Wid) get ot 
correct. In almost any subject we study, "learning the subject" 
consists in fact of learning the language or code of the subject. 
We explain some construct, then attach a name to it and if that name 
elicits the construct (or vice versa) we say we have learned it. For 
the above reasons then, the writer believes that principles discovered 
in the study of paired-associate learning may be applied to a much 
wider field of learning which may not at first be thought of as paired- 


associate in nature. 


III. DESIGN AND PROCEDURE OF THE STUDY 
In this study five teaching methods, two criterion tests and 
two time intervals of testing were studied. The levels of the three 


factors mentioned above are described below. 
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Method One (Partial Confirmation) This method presents the stimulus 


to the subject, elicits a response, checks the response and either 
feeds back the checked response requesting another response or flashes 
the word RIGHT on the C.R.T. If the subject uses all the alloted 
tries the correct response is flashed on the screen and he moves on 

to the next question. Each response is checked by retaining any 
character in the response which matches exactly with a character 

in the same position in the criterion response or any three or more 
consecutive characters in the criterion response. Those characters 
which do not match are replaced by hyphens. Then the checked response 
is flashed on the C.R.T. 

Method Two (Confirmation) This method presents the stimulus to the 
subject, elicits a response, checks the response then either flashes 
the word WRONG on the screen and presents the criterion response 

or flashes the word RIGHT on the screen and proceeds to the next 
question. 

Method Three (Prompt) This method presents the criterion response, 
presents the stimulus, elicits a response from the subject while the 
criterion response and stimulus are still on the C.R.T. 

Method Four (Multiple Choice) This method presents the stimulus, 

then presents six possible criterion responses from which the subject 
is required to point one for his choice of a response. If the selection 
is correct all the remaining distractors are erased from the screen 
and the word RIGHT is flashed on the screen. If the selection is wrong 
the word WRONG is flashed on the screen and the incorrect selection 


erased from the screen. The subject proceeds until he points to the 
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correct distractor. 

Method Five (Stimulus-Response) This method presents the subject 
with the stimulus and criterion response simultaneously. When the 
subject hits the space bar on the terminal the next pair of associates 
is presented. 

Test 1 (Constructed Response) This on test presents the stimulus 
(the English part) to the subject then requests him to type in his 
response. The response is checked and the next question asked. 

Test 2 (Multiple Choice) This criterion test presents the stimulus 
(the English part) and six distractors to the subject who responds 

by pointing to one of the distractors. 

Time One This is the time at which the first presentation of the 
criterion test is made and occurs immediately after the teaching 
methods are finished. 

Time Two This is the time at which the second presentation of the 
criterion test is made and is one week after Time One. 

The analysis of this study was divided into three distinct 

parts: 

Part one: a study of the differential effects of teaching 
methods over criterion tests and a study of the 
teaching methods on the criterion tests 

Part two: a study of the differential effects of the teaching 
methods over time (for each criterion test) and 
a study of the differential effects of time on 
the results of the teaching methods (as measured 
by Test 1 and Test 2) 


Part three: a set of planned comparisons to test various 
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7 
questions related to learning theory and applicable 
to this study. 

Random samples of students listed in the Central Volunteer 
Bureau's files were bussed to the University of Alberta in groups of 
ten. These ten were randomly assigned two to a method and five to 
each of the two criterion tests. Although some needed additional help 
most of the students learned how to use the terminals and proceeded 
through the program entirely under the direction of the computer. 
After proceeding through the teaching program and taking the criterion 
test of achievement the student waited one week (to the hour) before 
being bussed to the University for a second testing session with the 
same criterion test on which he had previously been examined. 

The computer program was entirely written by the author in 
the Coursewriter II language and was pretested before the experimental 


run. 


IV. DEFINITION OF TERMS 


Achievement in the context of this study will mean the number of paired- 
associates learned per unit time. If one learns more than someone else 
in the same amount of time he shall be considered to have achieved 

more. 

Amount ‘Learned shall mean the number of paired-associates correctly 
reproduced in the different criterion tests. It shall be referred to 

as and measured by the score on these criterion tests. 

Cathode Ray Tube (C.R.T.) the television-like tube on which the text 

of all the material presented to the student is displayed. Is often 


referred to as the screen. 
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8 
Computer Assisted Instruction (C.A.I.) is the name given to a teaching 
process which makes extensive use of a computer in presenting, testing 
and interacting with the student. 
Constructed Response refers to the type of response whereby the subject 
fabricates or constructs his answer without external cues. This is in 
contrast to the multiple choice type of response where the answer is 
already constructed for him and he has only to recognize the response 
from a set of alternatives. 
Information Feedback Interval refers to the interval of time between 
a student's response and the moment he is fed back information based 
on that response. 
Linear Programming refers to a program where all the subjects follow 
the same pattern of questions and all questions or frames are in one 
sequential pattern. There is only one route a student can follow 
from the beginning to end, the only variations among students being 
the time taken to complete the program. 
Method One (M1) refers to and is synonymous with the Partial Confir- 
mation Method of teaching. 
Method Two (M2) refers to and is synonymous with the Confirmation 
Method of teaching. 
Method Three (M3) refers to and is synonymous with the Prompt Method 
of teaching. 
Method Four (M4) refers to and is synonymous with the Multiple Choice. 
Method Five ees refers to and is synonymous with the S-R Method of 
teaching. 


Post Information Feedback Interval refers to the time between the 
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Information Feedback and the beginning of the next stimulus (or question). 
V. OUTLINE OF STUDY 


A review of pertinent literature is presented in Chapter II. 
A special chapter detailing the teaching methods used, how they relate 
to the literature and the questions they elicit will be labeled Chapter 
III, THE METHODS. Chapter IV consists of a detailed description of 
the design and the statistical procedures used in analysing the data. 
Chapter V includes the results of the statistical analysis and Chapter 
VI summarizes the study and presents the conclusions, limitations and 


implications for further research. 
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CHAPTER II 
REVIEW OF THE LITERATURE 


The classical explanation of how one comes to associate 
(learn) two words or concepts may be presented by paraphrasing Staats 
and Staats discourse on paired-associate learning (Staats and Staats 
1963). Two stimulii are presented to a subject who is required to 
associate them in such a way as to be able, upon presentation with 
one S, to respond with the other. The association of these S's may 
follow the tenets of classical conditioning. For example, in the 
common paired associations of words: 

1. One word acts as the stimulus (S,) which elicits the 
response (R,). (S,) may be the typewritten word dog; 
the R, would then be the subject saying, either overtly 
or covertly, ''dog". 


2. The other word (Sy) would also elicit a response (Ry) - 


Figure 4 


Se 3 
By R 


Simply by being presented together a relationship is set up 


y 


whereby S, elicits Ry which in turn produces a stimulus which will 


elicit Ry without the physical presence of Sy° 
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Figure 2 


Even though most text books will describe the process of 
associating two words in approximately the same way, many of the 
studies which investigated methods for doing this differed in their 
approaches to the problem as well as in their results. With respect 
to automated learning the programmed text book and mechanical teaching 
machine sagnletiaas seem to have done the most work in this area. 

Holland (p. 77 undated) in his booklet Research on Programming 
Variables describes two main methods of presenting "constructed response" 
(see definitions) material in classical linear programming sequences. 
They are the Prompting and the Confirmation methods. 

Prompting - means the criterion response term is shown before there 
is an opportunity for an overt response from the subject. 
Confirmation~sometimes called the anticipation method; means the 
subject is presented a stimulus, required to respond 
and then he is shown the criterion response term 
(checking). 

The Confirmation method advocated by Skinner (1958) has been 
found most efficient by researchers such as Gagne and Brown (1961). 

Angeli and Lumsdaine (1961) suggest there is no significant 
difference between the prompt and the confirmation methods of presenta- 
tion. Kopstein and Roshal (1955), Cook and Kendler (1956), Cook and 


Spitzer (1960), Kopstein and Shillestad (1961), and Stolurow and 
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12 
Lippert (1963) all report the superiority of prompting over confirma- 
tion. Holland (p. 75 undated) after extensive reading in this area 
said: 

None of the experiments found an unequivocal advantage for 
partial cueing over the standard anticipation (confirmation) 
method, though slight non-significant differences were common. 
The studies taken as a whole suggest a slight advantage for 
partial prompting over the anticipation method. 

An interesting study was done by Sidowski, Kopstein and 
Shillestad (1961). They defined six variations of the Confirmation 
and Prompt type of presentation: 

1. Confirmation - stimulus presented, subject required to respond 
overtly, response (correct answer) is then 


presented subject. 


2. Prompting ~ stimulus presented subject, response presented 
subject, subject required to respond overtly. 


3. Confirmation - stimulus presented, subject responds, response 
Prompt is presented, then subject is required to copy 
response. 
4. Prompt S-R - stimulus presented, stimulus and response presented, 


subject then required to respond. 


5. Confirmation - stimulus presented, subject responds, then 
S=R stimulus and response presented together. 
6. Simultaneous - stimulus and response presented simultaneously 
Observation but subject does not respond overtly. 


Number 2 and 3 methods were shown to be significantly superior 
on criterion tests to any of the three confirmation methods. Even 
simultaneous observation was significantly better than the confirmation 
methods. 

Other researchers, Hershberger and Torry (1965), and Faust 
and Anderson (1967), have refined the proupe method somewhat and report 


that it is most effective when the item difficulty is fairly high. 
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13 
They report that if the item is too easy the subject can get into the 
habit of merely copying without thinking about what he is doing. 
Other issues in programming besides those previously mentioned 
have arisen in the past and will be mentioned only as they relate to the 
above. 
tbe Constructed response (C=-R) verses Multiple choice (M-C): 
Both confirmation and prompting types of presentations are of the 
constructed response type. There is, however, some controversy 
over the desirability of this mode of response over the simple 
multiple choice. For one thing it is usually much more time consuming 
than the multiple choice mode. (In fact Sidowski, Kopstein and 
Shillestad (1961) had results which indicate the method not requiring 
an overt response was not less effective than those which did - 
hinting that overt responding may not even be necessary . Williams 
(1967) found that students taking a course designed with a combi- 
nation of constructed response (on technical items) and multiple 
choice (on general items) did significantly better on a C-R post 
test than did those taking a course designed with randomly mixed 
C-R and M-C type items. 

Ze Pacing: Two studies, Briggs, Ploshinski, and Jones (1959), Alter 
and Silverman (1962), found that programmer paced or student 
paced teaching programs seemed to have little differential effect 
on outcomes. 

3: The information feedback interval (I-F) (see definitions) and 
post information feedback interval (post I-F): I-F interval 
is pertinent only to the confirmation presentation while post 


I-F interval applies to both systems. Studies of operant condi- 
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14 
tioning suggest that the sooner reinforcement follows a response 
the quicker and stronger the S-R relationship will be established 
(Skinner, 1953). If it is assumed that most programs are written 
to minimize incorrect responses it may safely be said they should 
be given immediately after a response. Bourne and Bunderson (1965), 
and Boersma (1966) report no significant difference in learning 
when the I-F delay is varied. However, they had conflicting 
results with respect to post I-F delay. Bourne and Bunderson 
(1965) found delay of post I-F resulted in fewer errors while 
Boersma (1966) found no difference. 

Zinn (1967) in his article Languages for Programming Conversa- 
tional Use of Computers for Instruction mentions that over twenty 
computer languages and dialects have been developed especially for 
C.A.I. Several of these languages have the capability of feeding 
back to the student a partially corrected response and asking him 
to try again. Undoubtedly this feature has been used in many of the 
C.A.I. courses which have been written in the last few years. However 
this author has been unable to find any studies in the C.A.I. area 
which pertain specifically to the teaching of paired associates 
(this included a search of the latest edition of the ENTELEK Pro- 
grammed Instruction Guide List of Courses which lists thousands of 
research studies recently conducted in the C.A.I. and programmed 
learning field). The writer thus feels that a study into the different 
methods of teaching paired-associates, a process of fundamental importance 


to learning, may shed some light on how C.A.I. programs can be made 


more effective. 
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Although there seems to be little concensus on which method 
of teaching used by the programmed learning specialists is superior, 
an attempt was made to adapt four of the more popular methods to the 
IBM 1500 C.A.I. machine along with one method particular to a C.A.I. 
machine, in an attempt to establish the superiority of any one of 
these methods over the others. Bourne and Bunderson (1965) used a 
multiple-choice type test as their criterion measure. Others such 
as Kopstein and Roshal (1955) or Faust and Anderson (1967) used 
constructed response criterion tests. Because the two types of criterion 
measures are fundamentally different (the constructed response test 
is a recall task while the multiple choice test is a recognition 


task) it was decided to include a test-type of factor in this study. 
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CHAPTER III 
OUTLINE OF METHODS AND THEORY 
I. THE METHODS 


A brief outline of each of the five methods of presentation 
will now be given. When applicable, differences between the C.A.I. 
method (the computer adapted method) and the method as used by the 
programmed learning specialists will be outlined. A more detailed 
description of the C.A.I. methods and flow-charts for each method 
are described in the documentation of the computer program contained 
in Appendix C. 
ie Partial’ Confirmation Method (Method One) 

Classical 

This method was impossible with the technology available to 
classical programmers. 

C.A.I. 

The stimulus is presented on the C.R.T. and the subject is required 
to type in his response. After being compared with the correct answer 
and determined as incorrect those parts of the response which are 
correct are retained and displayed on the C.R.T. immediately after 
(in time) the word "WRONG" is flashed on the C.R.T. and then the 
subject is presented with the next problem. Each incorrect response 
is checked by retaining any character in the response which matches 
exactly with the character in the same position in the criterion 
response or any three or more consecutive characters in the response 


which match any three or more consecutive characters in the criterion 
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response. Those characters which do not match are replaced by hyphens 
and the checked response flashed on the screen. The student is then 
asked to respond again provided he has not used all the allotted tries. 
For responses with less than 50 per cent of the characters correct and 
no improvement over repeated tries a maximum of three additional chances 
are given. If he has more than 49 per cent of the word correct five 
chances are allowed. If a subject's current response is an improvement 
over his previous response he is allowed an additional response even 
though he may have been over the allotted three or five chance limit. 

For example: if the keyed response was Fluoroscope and the student's 
response was Flour the computer would respond first with the word 
"WRONG" then would display "Fl--r------ ". Thus the student is informed 
that he got three characters correct and that there are eleven characters 
in the correct response. He will then be asked to try again. 
Bi The Confirmation Method (Method Two) 

Classical 

A question or problem (the stimulus) is presented the subject 
in the form of printed text on paper. It may be a question, with a 
space provided for the student to write his answer (response) or it 
may be a statement in which a blank is located. The student is required 
to write a word (the response) in the blank which makes the statement 
true and sensible. He then advances the frame or turns the page to 
reveal the answer. Regardless of whether or not he is correct he 
moves on to the next question, or frame. 


C.A.IL. 





The student is presented with the problem on the C.R.T. He is 


required to type his answer (response) on a keyboard which automatically 
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18 
displays it on the C.R.T. screen immediately below the question. If 
he is correct, the word "RIGHT" is flashed on the screen to the right 
of the question and response, which are still displayed on the screen. 
If he is wrong the word "WRONG" is displayed, his answer is erased and 
the words "The correct answer is ~ " replace it. The 
sentence is positioned so that the correct response is immediately 
below the question word. 

Differences 

In the programmed method the student must realize that his answer 
and the keyed answer are the same or mean the same thing. All the 
C.A.I. method requires is that the student realize that the letters 
"RIGHT" mean he did what was asked of him and that the letters "WRONG" 
mean that he did not. The programmed method usually requires that the 
keyed answer be physically separated from the students' response. 
In the C.A.I. method the keyed answer is located immediately below the 
response and thus differences between the response and the keyed answer 
are more readily discernible. 
Ge Prompt Method (Method Three) 

Classical 

In this method both the required response and the stimulus word 
are presented together in the context of a sentence or a paragraph. 
The student has only to locate the correct word and copy it down in 
the space provided. The keyed answer is not revealed and the student 
never really knows whether or not he is correct. Regardless of whether 


the response is correct or wrong the subject goes on to the next problem. 
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C.A.I. 

Stimulus and response words are presented together a sentence 
or paragraph which is displayed on the C.R.T. The subject's response 
is displayed on the C.R.T. as he types it out. He then goes on to the 
next problem without getting any further feedback. 

Differences 

The only difference between these two methods is that the student 
writes in the answer in one case and types in the answer in the other. 
D. Multiple Choice (Method Four) 

Classical 

The stimulus is presented and the subject is required to check 
off (or write down the lable of) one of the several alternative responses 
presented to him. These alternatives may be in the form of a list of 
answers printed on a page or (in the case of the more advanced electro- 
mechanical teaching machines A.T.A. Outline (1962) ), several alter- 
native buttons. In the case of the programmed text books and mechanical 
teaching machines the frame is then advanced and the label of the 
correct response displayed. (The electro-mechanical teaching machine . 
could automatically determine whether or not the right button had been 
pushed and thus presented the course author with alternatives of having 
the student try again or going on to the correct answer and then the 
next problem. However the extreme cost limited the amount of use to 
which this machine was put and thus little is known of its usefulness 
or of its advantages over other programming methods.) 

CeAc1. 

This method is almost the same except for the fact that the text 


is presented on the C.R.T. and the student points to his choice of 
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alternatives with a light sensitive stylus (light pen). If he is 
correct the word "RIGHT" is flashed on the screen and all the incorrect 
responses are erased. After viewing the correct response for several 
seconds he then goes on to the next problem. If an incorrect distractor 
is pointed to, the word "WRONG" is flashed on the screen and that 
distractor is erased. The student then tries again. 

Differences 

The C.A.I. method has a consistent way of telling the student 
whether or not he is correct. i.e. By flashing the word "RIGHT" or 
"WRONG" on the screen. The classical method requires the student to 
match labels which may be. different for each problem. Feedback is 
immediate for the C.A.I. method. In the classical context the student 
must determine what the correct answer is by performing some further 
task (further to. responding), such.as. turning the page or turning 
to the next frame. He then must decide whether or not his answer is 
the same as.that, given... Then he may or may not interpret his correct 
response positively. 
Ee The Stimulus-Response Method (Method Five) 

Classical 

This method requires an electronic apparatus for pacing the pre- 
sentation of the stimulus and response together for a specified amount 
of time. (Sidowski, Kopstein, and Shillested, 1961). Presumably any 
machine like a tachistoscope or a timed film strip projector may be 
used. Nothing more than the two words or concepts to be paired are 
presented. There is no response required from the subject other than 


the reading of the associates to himself. 
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C.A.1. 


This method is exactly the same as the Classical approach except 
it is presented on the C.R.T. No overt response is required from the 
subject. Each pair is presented for five seconds then the next pair 
is presented for a similar time. 

Differences 


None 


THE CRITERION TESTS 


Ne Constructed Response (Test 1) 


This type of test requires the student to spell out his response. 
He must type in the complete answer. The actual format of presentation 
is very much like the stimulus presentation in the Confirmation method 
mentioned above. 
B. Multiple Choice (Test 2) 

All the student will have to do is point, with the light pencil, 
(see 'The Machine" Appendix B) to one of the alternative answers pre- 
sented to him. 

In the present study, to minimize problems due to lack of fami- 
liarity with the keyboard or light pen, an exercise section was included 


in all programs. 


LIMITATIONS OF CLASSICAL PROGRAMMING 

When one talks of a constructed response in regard to a pro- 
grammed text or mechanical teaching machine almost inevitably he is 
referring to the student or subject writing the required response 


in a blank space on the text. To correct his answer the student is 
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required to observe the keyed answer then determine whether or not 

his is the same (in the confirmation type of approach). If the student 
wrote an answer which was synonymous or which was an alternate he would 
be required to recognize the fact himself and even then he may not be 
really sure of the correctness of his answer. A student could con- 
ceivably think his answers are the same as the keyed answers while 

all the time they are wrong. His error would not be detected until 

the criterion tests (assuming the test was not self scored). 

The use of C.A.I., on the other hand, makes it possible to 
analyse each response and tell the student exactly whether he is 
correct, partially correct, or whether he is wrong, by simply presenting 
him with the word "correct", or with a pleasing picture such as a 
smiling face, a star, etc., or any other sign which cannot be mis- 
interpreted. In addition to this, the student can be informed 
whether or not his response was correct without telling him what the 
correct answer is (a feat which is impossible with constructed res- 


ponses as they have been used in the past). 


II. THEORY 

The following theory may lend some support to each of the 
methods being tested. In each case the computer adapted version will 
be the method refered to. 
A. PARTIAL CONFIRMATION 

This method differs from the classical confirmation method 
in four important ways: 

1. It eliminates punishment by never flashing the word "WRONG" 


on the screen. It could be considered continuous rein- 
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2a 
forcement (except when the student gets no letters of 
the response correct) because it always shows the student 
the parts of the response which he got correct and en- 
courages him to try again. 

2. It does not leave to chance (as does the confirmation 
method) the possibility that the student will try to 
find for himself the differences between his incorrect 
responses and the correct response. The differences 
are pointed out explicitly. 

3. The replacements for the parts of the response which the 
subject gets wrong are not given to him. He must think 
or guess to fill in the incorrect parts and presumably 
the extra effort required will further point out the 
differences between his original response and the correct 
response. 

4. The subject is required to correct his mistakes overtly 
whereas in the other methods he is only, at best, allowed 
to make a covert correction (a situation analogous to 
making a student write out a corrected spelling word 
as opposed to merely giving him the correct version 
and asking him to notice if his word is correct). 

Another feature of this method is that it allows the subject 
to learn at a slower or a more rapid rate depending on the individual. 
As he tries to fill in the erroneous spaces in his original response 
he may take three tries while filling in the three different errors. 
Each loop would emphasize the particular position (the one just 


corrected) which holds a particular item (the correct character 
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just entered). The subject can then devote his attention to the 


other errors ~ - as a group or individually, whichever the subject 


prefers or is able to manage. 


B. CONFIRMATION 

Skinner proposed a teaching machine, using the confirmation 
method of presentation, as a working example of his theories on 
operant conditioning. He viewed the act of getting a problem correct 
as a positive reinforcing event and thus a motivating force for the 
repeated elicitation of the same response when presented with an 
equivalent stimulus in the future. The instances in which the subject 
made incorrect responses and the occurrence of the word 'WRONG" 
could be viewed as punishment and thus would tend to suppress that 
action in the future (Hilgard, 1958). 

Since the correct answer is displayed after the subject 
produces an incorrect response the subject is able to discover dif- 
ferences between his response and the acceptable response. He should 
thus be able to make a clear distinction and adjust accordingly in 
future encounters with a similar stimulus. The subject, by looking 
at the eer ece answer, will determine where his answer is different 
from the correct answer and will know where to avoid a similar mistake 


in the future. 


Ge PROMPT 
This method seems to negate the need for reinforcement or 
even the need for feedback. Since the S and R are couched in the 


context of a sentence, the subject is required to differentiate the 
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given stimulus from all the other possible stimuli in the paragraph 
and then make the association with the correct response. For example 
a typical unit of learning using the prompt method is the following: 


In Latin a dog is a "canis" while the army is the 
"exercitus" and "puer' means boy. 


Army is 2 
First the subject has to find the stimulus which is similar 
to or the same as "army". To do this he must first differentiate 
between dog, army, and boy. While doing this he may reason "Dog 
is canis and dog is not army, therefore canis is not army. Army 
is exercitus and army is army, therefore the required response is 


exercitus.'' The set of units for the prompt method are shown in Appendix E. 


D. MULTIPLE CHOICE 

This method allows the student to differentiate between 
various possible answers before making a response. If he is correct 
he will receive a reward in the form of a "RIGHT" flashed on the 
screen. "Punishment" will be administered. if the response is in- 
correct (the word "WRONG" will be flashed on the screen). Since all 
but the correct response will be erased from the screen there will 
be no chance for the subject to discover differences between his 


response and the correct response. 


Es STIMULUS-RESPONSE 

If Staats and Staats (1963) are correct the mere fact that 
S and R are being presented together is enough to make the required 
associations. The advantage of this system is that the material to 


be taught can be presented much more frequently than in other methods. 
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Frequency of presentation may be more important in setting up stable 


associations than the more complicated adjuncts of the other methods. 


A COMPARISON OF THE THEORIES INVOLVED 

Several different theories and partial theories are dis- 
tributed among the five methods. Some are common to several methods, 
others are unique. Figure 3 shows the differences. 

The categories mentioned above were arrived at largely on 
an intuitive basis and are based to a certain extent on the following 
assumptions: 

1, That flashing the word "WRONG" on the screen acts 

as punishment. 
2. That flashing the word "RIGHT" on the screen acts 


as reward. 


III. QUESTIONS 
Assuming the statements made about the six features or 
categories mentioned above are true the following statements should 
be supported by this study. (The "questions" are presented in the 
affirmative for greater clarity). 

a (a) If the mean for the S-R method on both criterion tests is 
significantly (.10) larger than all the other methods, then 
some support will be given to the premise that overt responses 
are not necessary for the learning of paired-associates. 

(b) If the S-R mean is significantly lower than all other 
means on the Constructed Response Criterion Test (C.R. 
Test) and significantly higher than the inter-groups mean 


on the M.C. Test then support will be lent to the 
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UNIQUE FEATURES OF THE TEACHING METHODS 








Feature 


Requires overt responses 


Gives reward for correct 
response 


Provides opportunity to 
discover differences between 
subject's response and 
required response 


Gives feedback 


Gives punishment for 
incorrect responses 


Provides opportunity to 
discover differences between 
correct S-R associations and 
other incorrect associations 
before making a response 


anaes) 


Teaching Method 
Containing Feature 


Partial Confirmation (P.C.) 
Confirmation (C) 

Multiple Choice (M.C.) 
Prompt (P) 
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28 
theory that the methods requiring an overt constructed response 
showed up better because of the nature of the criterion test 
rather than any superiority in presentation and that overt 
responding may not be as important in learning as some theo- 
reticians would have us believe. 

If the mean for the P. method is significantly higher than 
the P. C., C., and M.C. methods there will be support for 

the theory that reward is not an important factor in learning 
paired-associates. (on either one or both criterion tests). 
Part (a) will also be supported if. the S-R method is superior 
to the P.C., C., and M.C. methods. 

If the means for either or both the P. and S-R methods are 
significantly higher tthan..those of the P.C., C., and M.C. 
then support.for. the. theory that feedback is not an important 
part of learning paired-associates. will be. lent. 

If the mean for the P. method is. significantly higher (on 
either or both criterion tests) than the P.C. and C. methods, 
then the theory that "pointing.out..differences before res- 
ponding is. better than pointing. out.differences after res- 
ponding" will be supported. 

If both the C. and M.C. method means show up significantly 
lower than all other methods, then the theory that "punish- 
ment has a dilatory effect on learning" will be supported. 

If the means for the P.C. and C. are significantly better 

(on either or both tests) than all the other means then the 
theory that “either reward or pointing out differences or 


both are important in learning. paired-associates will be 
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supported. 
6. If the P.C. mean shows up significantly better than the C., 
the pointing out of differences will probably be the deter- 
mining factor. Although, as mentioned earlier, the P.C. 
method probably emphasizes the reward factor more than does 
the C. method (this is because. of. not punishing, as is done 
in the C. method, as well as pointing out any little part 
of the student's response which is correct), the fact that 
the student does not get the word correct may be interpreted 
as punishment and thus even though he is told what he got 
right he may still. feel punished. 
The above suppositions or questions. have been presented as 
if there were only one criterion testing session. It is assumed 
the reader can make the necessary adjustments in the conclusions to 
be drawn when the criterion tests, given one week after the first, 
are administered. (i.e. In the one case we are talking about short 
term performance, in the other, long term performance or retention). 
The above questions are little more than guesses and will 
not be proven in this study. They are only offered as thoughts on 
"why" one thing or another should happen. The main purpose of this 
study was to find the best method (from among the five given) of 
teaching paired-associates, using only a computer to present the 
material to be learned. The following hypotheses were offered. 
IV. HYPOTHESES TO BE TESTED 
La That there are no significant differences between the criterion 


score means as measured by the multiple.choice test (Test 2) 
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30 
and the means as measured by the constructed response test 
(Test 1). 
There are no significant differences among mean scores for each 
of the five teaching methods as measured by both criterion tests. 
There are no significant differences between criterion mean 
scores for TEST 1 at time one and time two. 
There are no significant differences between mean scores for 
teaching methods for TEST 1. 
There are no significant differences between criterion mean 
scores for TEST 2 at time one and time two. 
There are no significant differences between mean scores across 


teaching methods for TEST 2. 
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CHAPTER IV 


EXPERIMENTAL DESIGN 


I. PROCEDURE OF THE STUDY 


In order to clarify the procedure of the present study a 


step by step outline is presented. 


tb 


Criteria were established for an operational definiton 
of learning. It was decided that amount learned per 
unit time would be the evaluation unit. Five levels 
of the teaching. method factor and two levels of the 
test factor were selected for Analysis. I. Analysis II 
required the five teaching levels and two time levels. 
A teaching program was then written in Coursewriter II 
which incorporated the unique characteristics of the 
five teaching methods and the two evaluation procedures 
(chapter 3) (See Appendix C). The twenty paired-associates 
that were taught are given in Appendix D. 
A trial group of subjects was tested on the program in 
order to: 

(a) Debug the program. 

(b) Establish a workable age level for the experi- 

mental subjects. 
(c) Establish time limits which. would allow adequate 
coverage of the paired-associates to be learned. 

Experimental. subjects were solicited.through playground 
supervisors and through the Central Volunteer Bureau in 
Edmonton. They were bussed to the IBM 1500 computer 


installation at the University of Alberta in groups of 


a) 
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32 
ten. An attempt was made to distribute the students evenly 
among the five testing methods and two criteria tests. 
Subjects were randomly chosen for assignment to any par- 
ticular method or test. 

General trial runs were made in order to debug the program 
proper. After the technical problems were removed strategy 
decisions were checked out and the following conclusions 
made. 

(a) That 11 and 12 year old children (the originally 
proposed experimental group) were too slow in 
developing the skills needed to efficiently 
operate the terminal units and thus should be 
replaced by 12 and 14 year olds. 

(b) That the teaching time (the time students spend 
on each method) should be lengthened from 20 
to 25 minutes in order to allow students to 
get through all the words in method one at 
least two times. 

(c) That the retesting session should be one week 
after the first. 

Originally one hundred and twenty students were contacted 
and agreed to participate in the program. Of these 87 
actually showed up for the first session and 68 attended 
both sessions. The dropouts, unfortunately, did not 
randomly distribute themselves over the five methods and 


two tests. Cell frequencies ranged from four to eleven 
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and this necessitated the use of an analysis program 
employing an unequal n's approach. (see Table 1) 

The time needed to get through method one was under- 
estimated and in most cases the students working on 
method one were unable to complete the twenty words in 
the time allotment. If one counts each time a word is 
presented, the number of words presented to individual 
subjects ranged from 11 to 281. 

Each subject went through four distinct steps in his 
first experimental run: 

(a) an untimed introduction to the computer 
terminal where the student learned to use 
the terminal. Each student spent as much 
time as he wished on this section and pro- 
ceeded only when he felt he was ready. 

(b) a section where all the twenty paired- 
associates to be learned were presented, 
each for a period of five seconds. (Appendix 
D) 

(c) a section where the student was taught the 
associates by one of the five different methods. 
He looped the twenty pairs for a period of 
twenty five minutes. 

(d) a section in which the subject was examined 
by one of the two criterion tests. (See 


Appendix C) 
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TABLE I 
NUMBER OF SUBJECTS IN 


EACH CONDITION 


SSS Sree torpor 
A a ALN TPES end 


METHOD TEST 1 TES412 
TIME TIME 7 TIME TIME Z, 
ONE TWO RETURN ONE TWO RETURN 

it i 5 yal 8 rE 88 

2 8 8 100 9 7 77 

3 8 5 63 12 11 92 

4 7 6 86 10 10 100 

5 8 4 50 10 6 60 
TONE 38 28 73 49 4] 83 
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7. One week after the first session the students were brought 
back for a second testing period on the same criterion 
test to which they had previously been exposed. This 
second session was also conducted entirely on the IBM 


1500 system. (Appendix B) 


IIT. ANALYSIS OF DATA 

Although one may consider this study to have three factors 
(method, type of test, time of testing) only two way analysis of variance 
tests were conducted since the three way interaction was not of interest. 
A. ANALYSIS I 

In analysis one the teaching factor and the "type-of-test" 
factor were used. ANOV 20 (D.E.R.S. 1968) was used. A two factor 
analysis with unequal n's and independent samples was performed by 
this program. Because of the dropouts only the first testing session 
results were used. Also, because the dropouts did not evenly distribute 
themselves it was decided that a posteriori tests on the teaching 
method means were not valid. 
Bs ANALYSIS II 

The second analysis was performed on the teaching method 
factor and the time factor. The University of Alberta's ANOV 23 
(D.E.R.S. 1968) was used. It employs a two factor design with repeated 
Measures and unequal n's. This analysis was used on both the multiple 
choice and the constructed response tests. 
ic ANALYSIS III 

A group of planned comparisons using the methods outlined by 


Winer (1962) were performed. These comparisons were designed in 
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accordance with the "Questions" asked in the previous chapter. The 
comparison weights used to test each question together with the number 
of the question being tested are given in Table VI. 


The formula used to calculate the F ratios is given below. 


Buz MS ithin cell 


Cy = contrast weight of level k 


A, = mean of the scores in level k 


Comparisons were conducted on simple effects rather than 
main effects. The denominator used would be the same as that used to 
test the main effects on the analysis of variance. The critical value 
used was aFe where df is the number of degrees of freedom for MS 
within. 

D. ANALYSIS IV 

In order to determine possible relationships among students' 
ages, intelligence and achievement, and in order to see if the more 
times a student is presented with a paired stimulus the more likely 
he is to learn the association, correlation coefficients were calcu- 
lated among the following variables: 

1. First testing session scores (both test 1 and test 2). 

2. Second testing session scores (both test 1 and test 2). 

3. Time to complete section on "how to use terminal". 

4. Number of times student was exposed to pairs. 


5. Age of student. 
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CHAPTER V 
RESULTS AND DISCUSSION 


I. ANALYSES 


A. ANALYSIS I 


The results of Analysis I are summarized in Table II. It 
indicates the test-type main effects to be significant at less than 
the .001 level and thus forces rejection of hypothesis one that there 
will be no differences between the types of criterion tests used. 
This of course was not unexpected. There is an obvious score ad- 
vantage to using a multiple choice criterion test. The cell means 
are shown in Figure 4. 

Method Main effects proved to be significantly different 
at the point .002 level. These effects are explored more fully in 
Analysis II. The main purpose for Analysis I was to check for an 
interaction between methods and test type. The results indicated 


that hypothesis of no interaction was not rejected. 


B. ANALYSIS II 

A summary of Analysis II is given in Tables III and IV. 
The average scores (expressed as percents) for the five methods and 
two tests at time 1 are shown in Table V. As can be seen from Table 
III a significant difference was registered in the "time" main 
effects (as measured by TEST 1) thus rejecting hypothesis 3. 


Both the methods main effects and the interaction failed to reach 


the acceptable level of significance thus hypothesis 4 was not 
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TABLE II 


TEST (1 & 2) 


SUMMARY OF ANALYSIS OF VARIANCE 





SOURCE SS df MS F iP 


METHODS (MAIN EFFECTS) 9339.5 4 2334.8 4.603% - 002 
TESTS (MAIN EFFECTS) 11147.1 1 11147.1 21.974 . 000 
INTERACTION 1990-3 4 498.0 ~981 423 


ERROR 39061.0 77 507.0 





* .1 level of significance accepted for this study. 
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Figure 4 


MEANS FOR TEST 1 AND TEST 2 


TEST 1 TEST 2 
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TABLE III 


TEST 1 


(CONSTRUCTED RESPONSE) 


SUMMARY OF ANALYSIS OF VARIANCE 








SOURCE OF VARIANCE SS df MS F P 
BETWEEN SUBJECTS 33403 26 1 

METHODS (MAIN EFFECTS) 6283 4 1570 Ther) 318 
SUBJECTS WITHIN GROUPS 27572 22 1253 

WITHIN SUBJECTS 2650 27 

TIME(MAIN EFFECTS) 748 1 748 10.825* .003 
METHOD-TIME INTERACTION 473 4 118 1.714 eLeZ 


TIME x SUBJECT WITHIN 
GROUPS 1520 22 69 








* .1 level of significance accepted for this study. 
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TABLE IV isi 


AMOR 4 


(MULTIPLE CHOICE) 


SUMMARY OF ANALYSIS OF VARIANCE 


NS PT HR SR NN 








SOURCE OF VARIATION SS df MS F P 
BETWEEN SUBJECTS 31556 40 
METHODS (MAIN EFFECTS) 4746 4 11867 1.648 -183 
SUBJECTS WITHIN GROUPS 25916 36 719 
WITHIN SUBJECTS 4050 41 
TIME (MAIN EFFECTS) 3 i 3 7039 844 
TIME x METHOD INTERACTION 393 ab 98 963 439 
TIME x SUBJECT WITHIN GROUPS 3675 36 102 

TABLE V 


AVERAGE PERCENT CORRECT ANSWERS 


FOR FIVE METHODS 
(Time 1) 








METHOD OF TEACHING 
CRITERION TEST M1 M2 M3 M4 M5 


ee 


CONSTRUCTED RESPONSE 23.00 3845099853. 5008 2405908, 51690 


MULTIPLE CHOICE 67.80 65:40) O78e6005957..508 M7750 
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rejected. 


Since the time main effects, methods main effects and inter- 
action effects were all non-significant for TEST 2 both hypotheses 


5 and 6 were not rejected. 
C. ANALYSIS III 


Table VI summarizes the results of the comparisons made 
among the various combinations of teaching method and testing method 


at time 1. None of the comparisons reached significance for Test 1 


or Test 2. 


D. ANALYSIS IV 

Correlation coefficients are reported in Table VII and 
show a significant negative correlation between the criterion tests 
and the speed with which the student went through the teaching section 
on how to use the terminal. All other coefficients turned out to be 
non-significant with the exception of the correlation between the 


first testing session and the second testing session. 


II. INTERPRETATION OF RESULTS 

Analyses of variance on TEST 1 and TEST 2 show conflicting 
results on the main effects of the time factor. One might expect 
that with time it is natural for students to forget and that both 
tests should show significant differences between the time levels. 
However, the two tests were different in a rather fundamental manner. 
The constructed response test (TEST 1) was a recall test, the student 
had to fabricate his response entirely from memory. The multiple 
choice test on the other hand, is a recognition test - all the 


subject had to do was recognize the response. 
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TABLE VI 


4 
SCHEFFE CONTRASTS OF CELL MEANS 
at Time 1 














COMPARISON QUESTION CONTRAST WEIGHTS Test 1 Test 2 
NUMBER TESTED M1 M2 M3 M4 M5 ANALYSIS ANALYSIS 
L -4 1 1 1 a 3.47 . 06 
2 1 -4 iL. ik 1 ee 40 
3 2(a) £ 1 -4 i nl 1.43 ZS 
4 il 1 iI -4 1 702 4.38 
Oeeeelay ibis Zb 1 1 1 1 -4 -90 1.44 
6 2c iv 1 -1 1/21 SL 2 woo 5.34 
Ue 5) -1 -1 2 0 0 Perey) 2.45 
8 5 el 2 —1el / 2a) 1 1 3.36 s22 
9 6 1 ~1 0 0 0 ae . 06 
10 4 1 -11/21 ss iL 2 .10 4.50 
TEST df for CRITICAL F VALUE 
denominator @=.10 @=.05 @=.01 
TEST 1 Ze 8.66" Bll. 28s 17524 
TEST 2 36 8.36 10.44 15.32 


SYMBOL USED IN TABLE VI 
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TABLE VII 
INTERCORRELATIONS 
VARIABLE 1 2 3 4 5 
Vee ist. cescting 1.00 86% -. 48% . 20 . 20 
2. 2nd. testing 1.00 -.45% -10 sks} 
3. Prelim. time 1.00 -.29 -.19 
4. No. of words 1.00 04 
5. Age of student 1.00 


N = 68 *critical r = .307 Ce=a. OL 
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The lack of interaction between the two tests (Analysis I) 
would tend to indicate that the major difference between how the 
various groups reacted to these tests is one of degree. It appears 
that one retains the ability to recognize much longer than one retains 
the ability to recall. 

Analysis II indicated that neither criterion test was able 
to detect significant differences among the teaching methods. If 
the plots of the means for each of the analyses in Figure 5 are con- 
sidered it can be seen that methods 3 and 5 are consistently higher 
than the other methods on the first testing. Perhaps, had the N 
been larger, the analysis of variance on TEST 1 and/or TEST 2 would 
have shown significant simple effects for the first testing sessions. 
Observation of Figure 5 reveals that the two best methods (3 and 5) 
on the initial testing session were also the two methods which resulted 
in the greatest drop in retention over time for TEST 1. Nevertheless 
none of these findings reached statistical significance. 

In order to get a better idea of the effects of time on the 
teaching methods the comparisons of Table VI were performed again 
on the simple main effects for TEST 1. The results of those com- 
parisons are reported in Table VIII. From Table VIII it can be 
shown that the F-ratios showing the differences of M3 and M5 
(comparisons 3 and 5) from the other methods, although not signi- 
ficant, do decrease somewhat from the first to the second testing 
session. The comparison of the difference between the combination 
of methods 3 and 5 and the other methods also dropped. One could 


speculate that if the time interval were extended (or that if more 
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FIGURE 5 46 
MEANS vs. METHOD BY TESTING SESSION 


*—* 1st session 
'— 7 2nd session 





M1 M2 M3 M4 M5 
N= 5 7 5 6 4 = 27 
(a) TEST 1 


M1 M2 M3 M4 M5 


(b) TEST 2 
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TABLE VIII 


COMPARISONS ON TEST 1 











TEST 1 
CONTRAST WEIGHTS lst session 2nd session 

# M1 M2 M3 M4 M5 F F 

A -4 1 1 i i 4.35 Zeol, 
Z 1 -4 1 1 1 5 NS 09 
3) HL: al -4 1 1 2.34 65 
4 i uh 1 -4 A 22 10 
5 iL i a 1 -4 1.44 42 
6 1 i -1 1/2 1 -1 1/2 5.24 49 
7 ih 1 -2 0 0 4.13 Pow 
a) coil en ny ya wat vl ik 4.10 2.34 
9 il -1 0 0 0 1.45 7) 
10 al -11/2 1 -1 1/21 ~42 .O1 
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levels of the time factor were added) the drop in M3 and M5 would 

have been even more significant and perhaps methods two or four would 
have been shown as superior. Since method one had the least drop 
between testing sessions it may have ended up on top had the time 
interval been extended far enough. However, a floor effect may be 
responsible for the small change in the scores for method 1. Perhaps 
it would have improved the design of the study had a group of students 
been given the tests immediately after being presented with the initial 
twenty words. The absolute effect of the teaching methods could 

then have been estimated. 

None of the contrasts listed in Tables VI and VIII was 
significant. However figure 5 indicates that method 3 was higher 
than all other methods with the possible exception of method 5. In 
view of the large drop off in attendance for method 5, one would 
suspect that method 3 may be superior to the others. 

It is interesting to note that in both TEST 1 and TEST 2 
the methods most like the criterion tests - method 1 for TEST 1 and 
method 4 for TEST 2 - resulted in the lowest means on those tests. 

Although nothing definite could be concluded from the above 
mentioned contrasts it should be pointed out that the procedures 
which traditionally have been considered important in learning (i.e. 
feedback, overt responding and reinforcement) and which were being 
tested for impact on criterion scores, (being tested in the sense 
that methods containing these characteristics were contrasted with 
those which did not) failed to show a significant effect on the 


subjects as indicated by their criterion scores. 
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CHAPTER VI 


SUMMARY, DISCUSSION, CONCLUSIONS, LIMITATIONS AND 


IMPLICATIONS FOR FURTHER RESEARCH 


I. SUMMARY 

The purpose of this study was to test the effectiveness of 
five different methods of teaching paired-associates on a compute- 
rized teaching machine. Effectiveness was measured by two different 
criterion tests - a recall test and a recognition test. Each of these 
criterion tests was used to measure the effectiveness of the methods 
over two different time periods - immediately after the teaching 
sessions and one week after the teaching session. The criterion 
used was amount learned per unit time. 

Samples of 12 - 14 year olds were randomly assigned to each 
of the teaching methods and 2 criterion tests. 

Six hypotheses were proposed for testing and are listed 
below: 

1. That there is no significant difference between the 


criterion score means as measured by the multiple choice 


test and the means as measured by the constructed response 


test - rejected. 

2. That there are no differences among mean scores for 
each of the five teaching methods as measured by a com- 
bination of both the criterion tests (Teaching methods 
main effects.) - rejected. 


3. That there are no significant differences between the 


criterion mean scores for TEST 1 at time one and time two. 


(Time main effects for TEST 1) - rejected. 
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4. That there are no significant differences among mean 
scores across teaching methods for TEST 1 (Methods main 
effects for TEST 1) - not rejected. 

5. That there are no significant differences between criterion 
mean scores for TEST 2 at time one and time two (Time main 
effects TEST 2) - not rejected. 

6. That there are no significant differences among mean 
scores across teaching methods for TEST 2 (Methods main 
effects for TEST 2) - not rejected. 

Although hypotheses four and six were not rejected, the lack 
of interaction between methods and testing procedures and the rejection 
of hypothesis two, would, upon inspection of the means in Table V and 
inspection of the F-ratios in Table VI, tend to favour Method Three, 
the Prompt Method. This is on the basis of testing at time one. 

There was also an indication that this same method resulted in the 
greatest loss in retention over time. The lack of interaction when 
testing hypotheses one and two indicated that the teaching methods 
did not act differentially on recall and recognition. Hypothesis 
five not being rejected indicated that the ability to recognize is 
retained over longer periods of time than is the ability to recall. 
There was also some indication that reinforcement, feedback, or even 
overt responding does not increase a person's rate of learning of 


paired-associates. 


II. DISCUSSION 


The fact that the two methods which supplied no feedback 
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(and in one case did not even require an overt response) seemed to 
produce the better results appears contrary to common opinion. This 
finding was not significant and may have been due to sample variation 
or may have been a result of the criterion chosen for this experi- 
ment - amount learned per unit time. Most paired-associate experiments 
use amount learned per presentation as the criterion, with little or 
no recognition being given to the amount of time taken. This criterion 
thus allowed the students in each group to view the pairs of differing 
number of times. That is, the number of times a student actually 
viewed one of the pairs was quite different for each of the methods 
used. Table IX indicates that methods three and five had the highest 
average number of words looked at. Since there were twenty pairs in 
all, it can be seen that many students in method one did not see the 
entire set of tasks. 

A one way analysis of variance performed on the number of 
words viewed gave a highly significant F-ratio. When a contrast 
between the average of methods three and five and the average of 


the other measures was performed an F-ratio of fifty was recorded. 


TABLE IX 


MEAN NUMBER OF WORDS LOOKED AT 





M1 M2 M3 M4 M5 





15.25 Boa/ > 59.40 43.12 165.00 
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These results would seem to indicate that the greater the number of 
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times a person views the paired-associates the higher would be his 
criterion score. However, the correlation coefficient between the 
number of words looked at and the score on the first criterion test 
turned out to be .20 and not significant. This would tend to indicate 
that something in these methods, other than the number of times a 
student views the pairs, was contributing to the superiority of the 
Prompt and S-R methods. It is noted from Table I that method 5, which 
was probably the most boring of the five methods tested, had the lowest 
return rate for the second testing session. Even though a method may 
produce good results initially as did this method, one must consider 
the motivational value of any technique finally decided upon. 

IIIT. CONCLUSIONS 

The results of this experiment, though not significant, would 
tend to indicate that the Prompt method of teaching paired-associates 
is the most effective, per unit of time spent learning, when one considers 
short term effects. If long term effects or retention are considered 
in relation to recall tasks, there appears to be little differential 
effect among the five methods tested. Recognition tasks on the other 
hand tend to remain more consistent over time and thus the Prompt method 
would be considered superior for this type of task also. (at least for 
short term retention tasks - i. e. one or two weeks.) 

IV. IMPLICATIONS FOR C.A.I. 

Keeping in mind that significance was not reached for any 
particular teaching method over either of the criterion tests taken 
individually but, that the methods main effect over tests was recorded 
as significant and that the highest mean recorded was for the Prompt 
method - one might say that at this point in time a Prompt approach 


to C.A.I. should be considered. Admitting that differences exist 
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between the learning of paired-associate and the higher order forms 
of learning, one may speculate as to how a "Prompt" approach to 
learning may be applied to a visual presentation on the C.R.T. The 
text of a lesson could be broken down according to significant ideas 
or concepts which are to be conveyed to the learner. These ideas 
would have to be small enough to be presented on less than about 
two-thirds of the C.R.T. - leaving one-third for the presentation 
of a question related to the textual presentation. The question 
would be phrased in such a way that it would focus the attention of 
the subject on the main idea being presented in the text. After 
responding the correct answer may or may not be presented to the 
subject. 

An alternate procedure useful for diagrams and/or text 
which cannot be presented in two-thirds of the screen would be to 
display the textual material on the film projector and use the C.R.T. 
exclusively for questions about that material. The function of such 
a procedure, as suggested by Cook and Spitzer (1960), seems to be to 
force the student's attention on the significant aspects of what he 
has just read. When a subject is novel a student has little infor- 
mation upon which to base his decisions about what is or is not 
important and could very easily skim past significant points which 


would be needed for understanding of future presentations. 


Vv. LIMITATIONS AND SUGGESTIONS FOR FURTHER RESEARCH 
This study was conducted using paired-associates. The 
extent to which one can generalize from the learning of paired-asso- 


ciates to the learning of higher order concepts is not entirely 
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understood. Even though methods three and five seem to work for the 
memorizing of pairs they may be completely useless when higher order 
learning is involved. An experiment employing the basic ideas out- 
lined for each of the methods mentioned above and attempting to 
teach higher order concepts would thus be very desirable. 

Only two levels of the time factor were used. An indica- 
tion that the initial effects of methods three and five are lost 
rather quickly was made earlier. The drop in methods three and 
five over one week brought their level down to that of the other 
methods thus forcing a conclusion that even over time these methods 
were at least as good as the others. However, had there been more 
levels of the time factor, tests for trend and other analyses may 
have indicated that over time the Prompt and S-R methods would in 
fact produce inferior results to one of the other methods. A study 
which extends the time levels may thus reveal important data relevant 
to the long term effects of different teaching methods. 

The criterion used for this study was number of associates 
learned per unit time. This criterion was chosen because, in the 
opinion of the author, it represents what in fact, if not in theory, 
is being emphasized in schools today. The lock-step method of 
advancing students emphasizes that a certain amount of material 
must be covered in a specified amount of time. This method also 
neccesitates the criterion measure be given relatively soon after 
the teaching sessions. This then accounts for the greater emphasis 
given to the first criterion tests in the write up of this document. 


It is recognized, however, that other bases for criterion measures 
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may be used which could be defended as well and perhaps better than 
those used in this project. The number of presentations of the 
associates could have been held constant and learning per unit number 
of presentations been the criterion. The student could have been 
allowed the choice of how long he would spend on the teaching method - 
he would have been instructed to stay on the teaching section of the 
program until he felt he knew the material. Initiation of the branch 
to the criterion test would be made by him. This method would be 
closer to the ideal of individualized instruction which is advanced 
by proponents of C.A.I. and possibly would have been a better cri- 
terion measure for this particular project than the one finally 
decided upon. It certainly approximates the real study situation 
where the determination and will-power of the student are real 
variables which enter into his actual achievement. 

At the time of the initiation of this study there were 
only two aspects of the IBM 1500 instruction machine which were 
operable - the C.R.T. and the film slide projector. Because of 
the need to limit a project of this sort to a manageable size only 
a small facet of the use of the C.R.T. was investigated in terms 
of the teaching methods which are possible on this machine. The 
use of graphics, (picture displayed on the C.R.T.) the use of the 
slide projector, the combined use of the projector and the C.R.T. 
are only a few of the areas which may be explored further. Now, 
a computer controlled audio unit is available on the 1500 and opens 
up a tremendous new field for investigation in learning. How does 
an auditory stimulus in paired-associate learning relate to visual 


stimuli? Comparisons of visual methods, audio methods, and audio- 
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visual methods of teaching can now be made and controlled to a degree 
never before possible. 

As mentioned in the introduction to this study it is the 
belief of this writer that the computer has a potential in the field 
of education and psychology which is limited only by the imagination 
of man. A popular argument against the use of a "machine" in teaching 
is that it is dehumanizing and will make robots of those taught by 
it. The experience of this author during the course of the past 
three years has led him to believe otherwise. It is his opinion 
that the computer is much less cold and inhuman than any textbook 
a student has ever been forced to interact with and is much more 
capable of transmitting the personality of the programmer than is 
any such textbook. Students have been observed interacting and 
gaining self esteem to a degree very comparable and perhaps greater 
than that which is possible in the present teaching situation in 


our schools. 
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THE MACHINE 


The terminals used by students consist of a C.R.T. for displaying of 


text, and a keyboard and light pen for the student to respond with. 
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APPENDIX C 63 


DOCUMENTATION FOR TRY 
Operator Information 
1. The name of the course is TRY and it has two segments. 
2. Only the standard and reverse dictionaries are used. 


3. The functions used are: 


(a) mv 
(b) ed 
(c) keyl 
(d) mec 
(e) es 
(f) sm 
4. Only one macro is called - - DUOO7. 


5. No films are required. 
6. No audio tapes are required. 
7. The estimated time for completion of the program is 50 minutes. 


8. All important response times are specified (when an ep is used 
as a pae the system time is used). 


9. This program should be self-explanatory. A section on how to 
use the terminals is included). 


10. The student will sign on as follows: 
on latin (student no.) 


11. No one should be required to supervise while running this program. 
12. There are no proctor messages. 
13. Performance recording will consist of 

(a) response times 

(b) counter values 


(c) text of responses 


14, The use of this program shall be restricted to those students 
whose names appear on a list which will be provided by the author. 


Programmer Information 
1. For flowchart see Appendix A. 


2. No new macros were developed for this course. The one macro 
used, DUOO7, is taken from the macro library. 






ss i 
é 
, 
a a ae ' 
= 2 - 
. a6 ‘ 
i. a) 
wo Bo ~The tA ate Cy 
rt - — —— ae a 
ov 
Cf] 
- 
\ : 7 bes. YR ef e@epe0 sn 
yor fas Bxebaste oA7 1 
4 > 
i y € OoLIS>ias 
7 
Ty, 
4 
i \ H 
é »' 
. 7 
5 4 “ii 
(@) 
- 
ii (3) 
- : bd — is = at of ge, amo vino 
e 
‘ ealit of 
» PS eM | it Ta 5 ou 
& ol on ato 
a i ; on Sais ‘* 
rl 
s tnetuogat LIA 
; # hd 2 SE Ss 2B 
~~ ea 
a oe & A’°TD> OIG elLdT 
= - i+ gee 
pag a e ee ets bay 
r . f 
* 7 mF | apie odT 
» 
‘ 4 ho 
out 
of Se2 xs ed bivgode sno oo 


va 


" : - > 


et = 

ep anihriopey earnersrzoltwd er 
, DT J 7 . —_ 

" ? i 





ogee 25 

— <. 
S me ie _ 7 ar 
I z a Fe - wits Sacer + ae Pa 2427203 ad iisda A&IPOTQ : : - on y 4 7 ” h 

7 re - < a | Sih dein P ‘ 4 7 

SB 4ad vd Bebivowa ed Ifiiw moidw gall a ao BSB ¢ ei 

. é ‘ 7s mn - . : a Pr 
i» a 7 





64 


3. All six buffers are used. 

4. All thirty counters are used. 

5. Switches zero through $22 are used. 
6. All six return registers are used. 


7. ‘The functions used are 


(a) ed 
(b) me 
(c) keyl 
(da) me 
(e) es 
(£) sm 
(g) mv 


8. There are no proctor messages. 
9. Response times are all specified. 


10. Where ZZ identifiers are used with the ep's no performance 
recording is done. 


11. No dictionary and/or graphic sets are used other than the standard 
and reverse dictionaries. 


12. No projector or audio tapes are used. 
13. Special Features 


(a) Three subroutines were developed in order to cut down on 
the volume of the program. The are: 


Macro 1 - This subroutine allows a multiple choice test to be pre- 
sented and marked with only the loading of buffers and 3 counters 
required of the author. If a different question is needed for another 
pass of the program, all or part of the text of the question may be 
changed by having the subroutine jump to a label at which the required 
instructions are located (see Appendix B). The subroutine will branch 
to the label, present the text of the question, then branch back to 
the routine and proceed with the rest of the processing. This sub- 
routine is used with Test 2. 


Macro 2 - This subroutine allows a student's answers to be analysed 
and partially fed back with dashes replacing mistakes in his response. 
It is used with method one, the Partial Confirmation Method. 


Macro 3 - This subroutine allows an author's text to be presented 
to the student in a multiple choice format. That is, a question 

may be asked of a student and several alternatives presented to him. 
If he points to the correct alternative he is rewarded. If he points 
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65 
to an incorrect alternative it is erased and the student is re- 


quested to try again. This subroutine is used in Method four, the 
Multiple Choice Method. 


(b) A method for randomly selecting methods and tests was de- 
veloped. Based on his student's number a subject is ran- 


domly assigned to one of the five treatments and one of the 
two criterion tests. 


Information for the Educator 


1. The purpose of TRY is to compare five different methods of 
teaching paired associates and two different criterion tests. 


2. The five methods of teaching are: 


(a) Partial Confirmation - that part of the students' answer 
which is correct is fed back with hyphens replacing the 
incorrect characters. 


(b) Confirmation - If students response is correct the word 
"RIGHT" is flashed on the screen. If it is incorrect the 
correct word is shown. 


(c) Prompt - The pair to be associated are presented in the 
context of a paragraph. A question is then asked and the 
correct answer must be determined by searching the para- 
graph. 


(ad) Multiple Choice - One of the pair of words to be associated 
is shown and the other word is presented as one of six 
alternatives. The student is required to find the other 
word. 


(e) Stimulus-Response - The two words to be associated are 
presented together. The student presses the space bar 
to get another pair. After going through one of the five 
treatments listed above the student will be given one of 
the following criterion tests. 


(a) A Constructed Response Test - a test where the 
student is required to type out the correct 
answer from memory. 


(b) A Multiple Choice Test - a test where the student 
is required to point to one of six alternative 
answers presented to him. 


3. The major source for information on methods (b) (c) (d) and (e) 
above were the writings of the early programmed learning special- 
ists. Method (a) above came from the writing in C.A.I. and isa 


uniquely computer based system. (i.e. it could not be approximated 


except on a computer. All the others had to be tried in pro- 


23 | 
-o3 «i tnefete oft Bae beense ek SP 
edt? ,nwo? botte di bea ok sadtvonipe 





ob sew atass fda abotien pn bisied@e Yineboa 108 
“fey ai szehdas © wsdmon s' oehyae abe gt palpi 


~ 


tO ehorijen tne slish evil staquosie? ei YA to epogspg eat ft 4 
27433 NGOliesian Jaezeti th ows Bas aggeiockes Rexzriaq 


2te ontkripAet 120 ebeyigem eve? eat .S 


AWhTLS itiehieds sit to traq tant = giifemsdtaos fetseee fa) 

aif? pr e: sugiloyd Ativ domed) Gat wy topzicm at ris trie 
haber . 

eon eins tourxoand 


cw at Joavroo ef eshogeat, sonaiaie 3 saigeeeiiae) ae 
wit 40Sssoon! .ai Ji BI 199192 = no Betesi= ee: “isord* 
Uwe ef Byov 29esttO 


“if oi Beata 7a stn bedpisoges sf oF 34849 ait - Sqnesd (5) 
rob bay esx Sojieeep A ,doetpeeed © to IxsJa00 = 
7 x Svie vd Bsniatassd ott Fat tered aSnsso5 

+ tie 1p 


Oe od Shacw 2c ears 32 2 ~ anion eigativm (bh) 
# Io 250 «sn Yetreesiq at Saow cease edd Bow aoe ak” ; 
J Oi? ov bevtupet Br onebide 42a7T tovirdnrupatis 
7 . Srow 


ys SUGlUQRRE ST oF Sshrow ON By ~ panogqe-i- ep lemisa. (a) 
Yo) steqe eld see@eenta cebhJsea ange ,toetss vod he tasaesq . 
ray i a? 16 ons din ae ih ts, eaieR se94h .zisy i230 Sag oo? q 
ta sao msviz of Iliyw poebioce aes ewxis feces! 2inaeteast : 
-a3es3 adfastets potweite? adf > 


= td S9ilw Jeod © - teaY sonal betestt a0. & tae) : 7 : 
s2es1e= sit tro eqyd of peericed Si Yaahute 





ae 


1 


eS 


66 
grammed text books in the past). 
The material in this program was designed for the grade six level. 


Execution time is about 50 minutes. 


This program is not really a course in the sense that it is trying 
to teach a lesson. It is designed as part of an experimental pro- 
ject to test different methods of teaching on the IBM 1500. 


No films or audio tapes are used. 


Keyboard responses are required exclusively in the 


multiple choice and 
test two. 


The student needs to know nothing about the operation of the 
terminal to take the course. 


A teacher or proctor is not needed for execution of the program. 


Performance recording is made of: 
(a) the number of correct responses made during the 
execution of a method. 
(b) the response times for each ep. 
(c) the score made on each test. 


No auxiliary materials are required. 

The programmer and author of this course is 
Donald W. Price 
4719 - 105 A Avenue 


Edmonton 80, Alberta 
466 - 7422 
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APPENDIX D 
THE PAIRED-ASSOCIATES USED 


IN THIS EXPERIMENT 


CAPITAL - CAPITIS 
BUILDING - AEDIFICIUM 
BRIDGE - PONTIS 
BOY - PUER 
CAMP - CASTRA 
ARMY - EXERCITUS 
DEEP - ALTUS 

DOG - CANIS 
FARMER  - AGRICOLA 
FATHER - PATER 
FIELD - AGER 
FIRE SRICINIS 
MAIDEN — VIRGINIS 
HAND - MANUS 
HEAD - CAPUT 
HILL Se CALLLS 
HOME - DOMUS 
HORSE - EQUUS 
HUNDRED - CENTUM 
LRON - FERRUM 
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APPENDIX E 
CONTEXTUAL PRESENTATIONS FOR METHOD THREE 


The capital of a country is called a capitis in our new language. 
If you were to take a journey to the capital it would be known as 
an itineris. The bus that you would travel on would be a toto. 

A capital is a i 


A bridge is a pontis, a market-place is a forum and a building an 
aedificium. 
A building is a : 


The house, or domus, of the boy, or puer, was located by a bridge 
which, in our new language is called a pontis. 
A bridge is called a : 


A family in our new language is called a familia while a boy is a 
puer and a girl is a pueress. 
A boy is a . ° 


The boy made camp, which is a castra, then proceeded to talk to the 


farmer or agricola who had just come across the field or ager. 
A camp is a . 


The army, which is called exercitus, left the city to chase a dog, 
a canis. The dog was caught in a field, an ager. 
The army is called 


A boy, who is a puer, jumped down a deep, or altus, well when he 
saw the fire, or ignis coming. 
Deep means ° 


A farmer or agricola looked across the field or ager and saw a dog 
which is a canis in our new language. 
A dog is called a . 


Hand is called manus in our new language while farmer is known as 
agricola and hundred is centum. 
A farmer is an ey 


A father who is called a pater once went up a hill, or callis, to 
build himself a home or domus. 
A father is called : 


A man, who is a manus, put his hand to his caput. That is, he put 
his hand to his head. He did this as he walked across the ager 

or field. 

A field is a 
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The horse, or equus, could run like wild fire and he belonged to 
a maiden. That is to say he could run like wild iginis. The 
horse belonged to a virginis. 

A fire is called - 


The boy had a manus of iron. Or, to put it another way he had a 
hand of ferrum. He liked to dig deep holes. That means he like 
to dig altus holes. So he tried to join the exercitus or army. 
A hand is known as_ 4 


The meaning for the word caput is given in the last sentence of 
this paragraph. When a dog or canis attacks a camp it is said to 
have attacked a castra. It means head. 

A head is a . 


The army, or exercitus, advanced up the callis to attack the 
agricola, or farmer. The farmer was on the hill that the army 
advanced up. 

A hill is a 


Father, on the way domus, fell in an altus or hole. We could also 
say, pater on the way home fell in a deep hole. It cost him one 
centum ($100) dollars to have it fixed. 

A home is called : 


Pater bought an equus for one hundred dollars. What father bought 
was a horse. It had a deep, or altus, brown spot on its caput. 

It had a spot on its head. 

A horse is an . 


One hundred delegates from the capital, or capitis, city voted yes 
to have the aedificium made of iron. Centum delegates voted yes 
to have a ferrum building. 

Hundred means 


A domus made of ferrum would make any virginis feel better than if 
she had to live in a field, or castra. That is, a maiden prefers 
a house made of iron to living in a field. 

Iron means 


The aedificium caught on fire and left the virginis homeless. It 
cost one hundred, or one centum dollars to replace the building 
which caught on iginis and left the maiden homeless. 

A maiden is a 
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